The Whole World in Your Hand: Active and Interactive Segmentation

Object segmentation is a fundamental problem in computer vision and a powerful resource for development. This paper presents three embodied approaches to the visual segmentation of objects. Each approach to segmentation is aided by the presence of a hand or arm in the proximity of the object to be segmented. The first approach is suitable for a robotic system, where Figure 1: The platforms. the robot can use its arm to evoke object motion. The second method operates on a wearable system, viewing the world from a human's perspective, with instrumentation to help detect machines to visually perceive the extent of maand segment objects that are held in the wearer's nipulable objects. Furthermore, we show that the hand. The third method operates when observing object segmentations that result from these metha human teacher, locating periodic motion (finods can serve as a powerful foundation for the ger/arm/object waving or tapping) and using it development of more general object perception. as a seed for segmentation. We show that obThe presence of a body changes the nature of ject segmentation can serve as a key resource for perception. The body provides constraint on indevelopment by demonstrating methods that exterpretation, opportunities for experimentation, ploit high-quality object segmentations to develop and a medium for communication. Hands in parboth low-level vision capabilities (specialized featicular are very revealing, since they interact diture detectors) and high-level vision capabilities rectly and flexibly with objects. In this paper, we (object recognition and localization), demonstrate several methods for simplifying visual processing by being attentive to hands, either